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Summary 

Historically, the development of advanced automation for 
air traffic control in the United States has excluded the 
input of the air traffic controller until the end of the devel- 
opment process. In contrast, the development of the Final 
Approach Spacing Tool (FAST), for the terminal area 
controller, has incorporated the end-user in early, iterative 
testing. This paper describes a cooperation between the 
controller and the developer to create a tool which incor- 
porates the complexity of the air traffic controller’s job. 
This approach to software development has enhanced the 
usability of FAST and has helped smooth the introduction 
of FAST into the operational environment. 

Introduction 

The development of an automation system for assisting 
terminal area air traffic controllers in efficiently managing 
and controlling arrival traffic has long been an objective 
of researchers and engineers. A fundamental issue for 
researchers in the development of such automation tools 
is to build functionalities and user interfaces that enhance 
the air traffic controllers’ ability to perform well in their 
job. Early efforts in the automation of terminal air traffic 
control was presented by Martin and Willett (1968). Their 
system provided speed and heading advisories to con- 
trollers to help increase spacing efficiency on final 
approach. Although traffic tests of the system showed 
an increase in landing rate, controllers found that their 
workload was increased and rejected the system (Martin 
and Willett, 1968). An examination of the concept sug- 
gests that while some aspects of the design were sound, 
its acceptance was limited by the technology of the time 
period, especially the lack of an adequate controller 
interface. More recently, however, several automation 
systems have found their way into operational use in 
Europe due in large part to the introduction of modern 
computer processing and interfaces, and also because of 
more careful design approaches (Volckers, 1990; Garcia, 
1990). In addition, recent real-time simulation studies 
have confirmed the potential for increasing landing rates 
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with the assistance of active advisories for controllers in 
the terminal area (Davis et ah, 1991 ; Credeur et al., 

1993). 

A candidate system for the automated management and 
control of terminal area traffic, referred to as the Center 
TRACON Automation System (CTAS), is under devel- 
opment at NASA Ames Research Center in collaboration 
with the Federal Aviation Administration’s (FAA’s) 
Terminal Air Traffic Control Automation Program 
Office. The elements comprising the CTAS are the 
Traffic Management Advisor (TMA), the Descent 
Advisor (DA), and the Final Approach Spacing Tool 
(FAST) (Erzberger et al., 1993). The advisories generated 
by these tools assist controllers in handling aircraft 
arrivals starting at about 200 n.mi. from the airport and 
continuing to the final approach fix. Recently, the ele- 
ments of the CTAS system have been evaluated in a 
series of real-time simulations at NASA Ames Research 
Center and in field testing at facilities serving the Denver, 
Colorado, and Dallas/Fort Worth, Texas, areas. 

This paper describes a cooperative effort between 
software developers, human factors specialists, and air 
traffic controllers to develop FAST. The main function of 
FAST is to provide landing sequence, landing runway 
assignments, and speed and heading advisories that help 
controllers manage and control arrival traffic and achieve 
an accurately spaced flow of that traffic onto the final 
approach course (Davis et al., 1991; Davis et al., 1994). 
This paper emphasizes the role of the air traffic controller 
in FAST development, from providing direction into the 
controlling strategies incorporated into the FAST algo- 
rithms. This paper also describes the use of the Controller 
Acceptance Rating Scale to track the controllers’ perspec- 
tive of the overall system through the development of 
FAST. 

Controller Input into FAST Development 

The CTAS development process has differed from more 
“traditional" approaches to software development by 
incorporating the expertise of the end-user at the begin- 
ning stages of development. Air traffic controllers have 
been involved in simulations at NASA Ames Research 



Center, the FAA Technical Center in Atlantic City, 

New Jersey, and in evaluation activities at the operational 
facilities into which the initial deployment have been 
targeted. The extensive involvement of controllers 
increases their understanding of the software and engi- 
neering constraints, and helps them to focus their exper- 
tise and provide input that is maximally useful to the 
software developers (Sanford et al., 1995). In addition, 
early end-user involvement also helps to integrate human 
factors issues during design, which can help improve 
system usability (Small, 1994). Larger questions of the 
suitability of a system, or how well the system provides 
for the users’ problem-solving requirements (Harwood 
and Sanford, 1993), can also be addressed with early end- 
user involvement. The development, and ultimate demise, 
of the Advanced Automation System (AAS) provided 
numerous examples of design problems that could have 
been ameliorated with earlier controller input. Small 
(1994) describes how controllers involved toward the end 
of development were inappropriately focused on inter- 
face, rather than operational elements of the design. In 
addition, controller involvement late in the development 
process created an environment in which the controllers 
and the developers saw each other on opposite sides. 

This made reaching compromises much more difficult, as 
developers did not understand the operational needs of the 
user, and the controllers viewed development constraints 
they encountered as arbitrary (Small, 1994). 

FAST simulations have been conducted at the Ames 
Automation Laboratory, where the software was origi- 
nally developed and tested. Controller participation in 
these simulations, and other design activities, has helped 
to shape the requirements, test the software, and provide 
input into human factors issues as well as insight into 
controller strategies to reduce workload and increase 
throughput efficiency. In addition to a pool of local, 
retired controllers, controller personnel from the Dallas/ 
Fort Worth (DFW) TRACON, where the initial FAST 
testing and deployment will occur, have been evaluating 
FAST. DFW facility representatives, including Union 
members and managers, have been encouraged to 
participate in the software evaluations. In total, three 
groups of controllers were incrementally involved in the 
FAST simulations. 

The simulations consisted of traffic scenarios displayed 
on radar screens with FAST advisories added to the radar 
display interface. The traffic scenarios were created from 
recordings of live radar traffic from DFW 7 . Two feeder 
controller positions (East and West) and three final 
controller positions (for the three arrival runways) were 
used for simulating DFW operations. Each arrival sector 
controller worked from a separate radar display position. 
The level of traffic during the 1 to 1 .5 hour simulations 


was approximately 80 to 100 arrival aircraft per hour, 
reflecting the actual “rush” durations and traffic levels at 
DFW. Controllers were provided with headsets through 
which they issued commands to pseudo pilots, located in 
another part of the laboratory. The controllers were able 
to accomplish all of the basic entries and inputs into the 
system that would be expected in the real facility, such as 
taking handoffs and changing runway assignments. 

Developers monitored the simulations in the laboratory; 
following simulations, debriefings were held to allow the 
controllers and developers to discuss key decisions made 
during the simulations, and explain any observed prob- 
lems. All of the simulation outcomes were recorded for 
later review, and the debriefing sessions often included a 
replay of the simulation. 

FAST has also been evaluated by controllers using a 
technique known as “shadowing.” Shadowing involved 
viewing the live operational activities in real time, with 
FAST operating in the background, superimposing its 
advisories on an auxiliary computer display. Shadowing 
enabled controllers and developers to view the effects of 
the controllers* current procedures and how the traffic 
outcome was influenced by the presence of FAST 
advisories. Shadowing activities occurred in a testing 
environment near DFW, where radar communications 
were also monitored to provide a more complete picture 
of the traffic flow and decision-making activities. 

Phases of Controller Involvement 

Figure 1 shows the development of FAST through 
varying levels of development milestones and controller 
involvement. This paper concentrates on the three phases 
of controller involvement in simulation, from the initial 
software development to the preparation for the opera- 
tional test. 

Three different controller teams participated in the three 
levels of simulation Fidelity. Each group of controllers 
provided increasingly more refined information in their 
evaluations. The first level, which required the initial 
assessment of new functionality and new displays, 
utilized the expertise of a group of local, retired con- 
trollers and pilots. The second level focused on expanded 
functionality and site-specific issues and incorporated the 
participation of a small cadre of controllers, or System 
Development Team (SDT), who represented the DFW 
facility. The third level increased the amount of feedback 
and input into refining the FAST functionality and deter- 
mined levels of acceptability for limited field testing, 
utilizing the expertise of a larger group of controllers 
from DFW. 
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Figure 1. Controller involvement in the context of FAST 
development milestones. 


Phase I: On-Site Controllers 

The first level of FAST development involved creating a 
system prototype in which new functionality was first 
tested and graphical user interfaces were generated. 
Simulations were conducted several days a week with 
on-site controllers, most of whom were retired from 
regional facilities in the San Francisco Bay Area. Feed- 
back from this early testing helped direct the iterative 
development of FAST. 

The simulations with the on-site controllers enabled the 
developers to gauge the software’s performance under 
basic, day-to-day testing conditions. The on-site con- 
trollers provided an indication of how air traffic control- 
lers as a whole would react to the displays, controlling 
strategies, and advisories incorporated into the software. 
From a human factors standpoint, they provided the most 
input at the level of FAST’s usability, determining if 
basic information could be extracted under simulated 
operations. They also provided some input on suitability 
issues, evaluating whether the information provided by 
FAST was appropriate for its intended use. In addition, 
the researchers benefited from on-site controller partici- 
pation in that they were able to conduct studies under 
fairly well-defined conditions to determine the impact of 
FAST on controller workload. These studies showed that 
FAST reduced the level of self-reported workload by the 
air traffic controller (Lee et al., 1995) as well as reducing 
the number of commands issued to aircraft (Slattery et al., 
1995). 


It was clear, hoWevdi, that the development of FAST 
functionality soon required more specific input from the 
DFW TRACON controllers, once basic concepts of air 
traffic control (ATC) were incorporated in the software. 
The on-site controllers’ input into basic ATC concerns 
and attitudes was often accurate, but their understanding 
of the operations at the DFW TRACON was limited. As a 
result, the SDT was brought into the development 
process. 


Phase II: System Development Team 

The second phase of FAST development incorporated 
the expertise of a cadre of three controllers, one area 
supervisor, and one training specialist from the DFW 
TRACON. These controllers formed the SDT. The SDT 
provided the developers with guidance about the specific 
procedures of the operational environment and an under- 
standing of operations under the high traffic levels that 
are common to DFW. The SDT participated in formal 
software evaluations at NASA Ames and the FAA 
Technical Center approximately every 6 to 8 weeks, 
depending upon milestones in the software design. 

Split of FAST functionality- One of the SDT’s pivotal 
inputs into FAST development was to split FAST 
functionality into two parts. FAST was originally con- 
ceived to provide a suite of advisory information for the 
TRACON sector controllers. This information would 
enable the TRACON controller to efficiently sequence 
and space arrival traffic by providing sequence and 
runway advisories. In addition, speeds and headings were 
provided to efficiently meet the sequences and runway 
assignments. Following an early evaluation, the SDT 
recommended that FAST be split into “passive” and 
“active” stages. Passive FAST was the portion of the 
software that provided only the sequence and runway 
advisory information and Active FAST was the portion 
where speed and heading advisories would be added. 
Together with the developers, the SDT decided that 
Passive FAST would be tested first. 

In the early months of FAST development, the SDT was 
very cautious toward FAST and their recommendation to 
test Passive FAST first was directly attributable to this 
cautious attitude. It is possible that when the SDT first 
began to evaluate FAST, the utility of the FAST advi- 
sories was not fully developed, and hence the benefits 
were not as obvious as they have become. However, the 
SDT’s acceptance of FAST increased significantly in the 
three years of their involvement in FAST development, 
and as a number of their concerns outside the software’s 
capability (described below) were raised and addressed. 
This suggests that their initial apprehension toward 
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developing full FAST capability was due to issues in 
addition to the immaturity of the software* 

There were a number of reasons for the SDT’s initial 
hesitancy. First, from the facility’s perspective, the 
involvement of the SDT members could have implied 
that the facility’s controllers were inefficient and that the 
abilities of the controllers as a whole were being called 
into question. It was easy to draw the conclusion that the 
development of ATC automation would prompt the FAA 
to reduce the controller workforce in response. Second, 
the SDT was clearly responsible for the outcome of 
FAST; thus the SDT’s reputation as seen by the facility 
would be at stake. Third, the SDT had concerns about 
liability. The FAA could decide that FAST should dictate 
commands to the air traffic controller, yet the controller 
(and not the software) would be ultimately responsible for 
the consequences of the FAST advisories. Finally, the 
SDT was concerned that FAST would automate the 
interesting aspects of the controller’s job and thus reduce 
job satisfaction. 

These concerns were not all obvious in the beginning of 
the development process. Some of the concerns the con- 
trollers discussed directly; other concerns were elicited 
after many months of development. The developers 
learned to provide reassurance to the controllers in addi- 
tion to promoting the potential benefits of FAST. Also, 
the developers earned the trust and respect of the SDT 
through demonstrating a thorough understanding and 
appreciation of ATC in general, and DFW operations in 
particular. This knowledge helped the controllers and the 
developers work together to address the SDT’s concerns, 
resulting in changes to the software, or making compro- 
mises when software changes could not be accomplished. 

SDT input into FAST algorithms- The SDT controllers 
hoped that FAST could provide a reduction in controller 
workload. However, it quickly became clear that while a 
basic element of workload was the number of aircraft for 
which a controller was responsible, other variables, such 
as the types of aircraft involved, the characteristics of the 
sector’s airspace, and the amount of coordination between 
positions, had to be considered. Consequently, the SDT 
showed that some FAST advisories, while intended to 
reduce controller workload by reducing delay, sometimes 
made the traffic situation more complicated from the 
controller’s perspective. 

The SDT also pointed out situations where FAST 
algorithms produced procedural obstacles and con trolling 
strategics that were considered unfamiliar or unfavorable. 
These issues required the developers and the controllers 
to work together to determine how the advisories were 
perceived and what could be done to minimize the impact 
on coordination and workload. In some cases, the SDT 



Figure 2. Possible routes for Southeast cornerpost arrival 
traffic. 


discovered that the unfamiliar strategies FAST suggested 
were beneficial ones. For example, FAST would propose 
that aircraft from the Southeast cornerpost fix be vectored 
over the top of the airport to a West-side runway. The 
current procedures would have specified a different route 
to the West-side, or would have directed the aircraft to the 
East-side runway, as shown in figure 2. FAST’s over-the- 
top vectoring was not typically favored by the facility, but 
the SDT found that this routing could provide benefits to 
their overall operations by decreasing the workload of the 
East runway controller. 

Shadowing observations were a key turning point in the 
SDT’s assessment of the software; they became an 
evaluation environment in which the benefits of FAST 
advisories became very evident. Figure 3 depicts a situa- 
tion observed in many cases at DFW where the arrival 
rush side of the airport would be overloaded. More 
experienced controllers would have balanced the runway 
loading better than that seen in figure 3, with vectoring 
strategies similar to those described earlier. By comparing 
the outcomes of FAST advisories to actual facility opera- 
tions, the SDT and the developers could see where FAST 
could provide increased flow and efficiency as well as 
help determine where FAST needed to adjust for facility 
procedures. 

After two years of SDT evaluations, it became clear that 
the SDT was providing input that was native to their own 
style of traffic control that was not necessarily representa- 
tive of DFW as a whole. As more advanced development 
issues were encountered, further input from the facility 
would be needed. In addition, FAST development was 
nearing a phase in which an operational test would be 
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Figure 3. Shadowing observations demonstrating uneven 
runway baiancing. 

required. In order to conduct such a test, a larger group, 
or assessment team, of DFW controllers needed to be 
assembled and made proficient in using FAST. 

Phase III: Assessment Team 

The Assessment Team was composed of six controllers 
(three from each specialty) and two area supervisors from 
DFW TRACON. The Assessment Team participated in 
simulations at NASA Ames and the FAA Technical 
Center and will be participating in the upcoming Limited 
Operational Demonstration at DFW. The Assessment 
Team brought a wider range of skill, expectations, and 
viewpoints to the FAST development process. In their 
initial introduction to FAST, some of the Assessment 
Team members were pleasantly surprised and impressed 
that FAST could increase their throughput capacity 
without making them feel taxed. Others felt that they 
operated quite well without the need for added auto- 
mation. As with the SDT, the developers had to work 
with the Assessment Team to come to an understanding 
of where the benefits of FAST’s advisories and strategies 
could be realized. While the Assessment Team also 
introduced new controlling strategies not previously 
encountered, they also saw the benefits of changing their 
controlling strategies in the operational environment 
based on what they had seen in FAST simulations. 

The Assessment Team’s main goal was to determine 
when FAST would be ready for a limited operational test. 
But because developing ATC automation with active 
controller input was a new process within the FAA, there 
were no established guidelines to help determine system 
readiness and acceptability. Furthermore, the benchmarks 
of acceptability from the developer’s perspective did not 


sufficiently address the controller’s sense of workload. It 
was clear that qualitative measures of workload and 
acceptance had to be assessed along with quantitative 
measures of reduced delay and increased runway 
throughput. 

The Controller Acceptance Rating Scale (CARS) 

Based on the need for a measure of acceptability, the 
Controller Acceptance Rating Scale (CARS) was devel- 
oped by the human factors specialists (fig. 4). The CARS 
was an adaptation of the Cooper-Harper Rating Scale for 
pilot assessment of handling qualities of aircraft (Cooper 
and Harper, 1969). CARS is still being assessed and 
validated at NASA Ames. 

The original Cooper-Harper scale measured the per- 
formance of the pilot and the vehicle working together; 
this merging of the pilot and the vehicle defined the 
system being evaluated (Harper and Cooper, 1986). In 
adapting this scale to the ATC environment, this concept 
of a system including the user, the software, and the hard- 
ware was preserved. In addition, changes were made to 
the scale layout itself: compared to the original Cooper- 
Harper scale, the scale was reordered so that a rating of 
“1” was the worst performance level, and “10” was the 
highest performance level. This change was made to 
reflect that a lower number was to be associated with a 
lower, less acceptable rating, and the higher number was 
to be associated with a higher, more acceptable rating. 

CARS was intended to provide the Assessment Team 
with a means for determining how well they and the 
FAST software performed together in controlling traffic 
in simulation. Following a simulation, each of the 
Assessment Team controllers viewed the scale, produced 
a rating, and gave an explanation for the rating. In addi- 
tion, they were asked to provide a confidence indication 
of their rating, which could incorporate issues of the 
fidelity of the simulation, the amount of information they 
had available to make their rating, and other issues that 
might be external to the software advisories. 

The four main rating levels described by CARS were 
controllability, tolerability, satisfaction, and acceptability. 
The controllers started at the top of the CARS diagram 
and first determined if the system was controllable. An 
uncontrollable system would imply safety violations such 
as near misses, collisions, or an inability to maintain 
separation. Such a system would require mandatory 
improvements. 

If the system was rated as controllable, the controller then 
assessed the next level: tolerability. If the system was 
rated as intolerable, this implied major deficiencies and 
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Only mildly unpleasant deficiencies. System is acceptable 
and minimal compensation needed to meet desired 
performance. General agreement with advisories. Use of 
passive FAST enhances performance. 

8 

Negligible deficiencies. System is acceptable, and 
compensation not a factor to achieve desired performance. 
Overall agreement with advisories. Significantly enhances 
performance. 

9 

Deficiencies are rare. System Is acceptable, and controller 
does not have to compensate to achieve desired 
performance. Nearly 100% agreement with advisories. 

10 


Figure 4. Controller Acceptance Rating Scale. 


an inability to achieve adequate performance with toler- 
able workload levels. If the system was rated as tolerable, 
this implied a reasonable workload with manageable 
deficiencies, and the controller then assessed the next 
level: satisfaction. An unsatisfactory rating implied that 
the deficiencies warranted improvement. If the system 
was considered satisfactory, however, the controller then 
rated the level of acceptability the system was able to 
achieve in the simulation. A confidence rating was 
obtained following the numerical rating. The Assessment 
Team was quite receptive to CARS. 

To date, CARS has been used for one Assessment Team 
evaluation, in which the controllers participated in seven 


simulations with varied runway configurations typical to 
DFW operations. Each controller who participated in a 
simulation rated the system based on his own experience 
working traffic from his particular sector position. 

Because CARS data have been collected from only one 
series of simulations, results are not presented here. In 
subsequent simulations and operational testing, the devel- 
opers intend to continue to use CARS in order to track the 
progress of the software evaluation by the controllers. 
CARS has indications of being a useful measurement 
device, and its validation will be useful to other ATC 
software development processes. 
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The Benefits of a Cooperative Design 
Approach 

Closely linking the air traffic controller with the aero- 
nautical engineer in developing ATC software helps to 
produce a system that will capture as much of the actual 
complexity of ATC as is possible prior to final field 
deployment. The process for incorporating air traffic 
controllers in the development of FAST allowed the 
FAST engineers to achieve a greater understanding and 
appreciation of the overall task of air traffic control, and 
helped them to create the most effective algorithms for 
generating efficient sequences and runway assignments. 
This process also allowed the controllers to become 
involved in understanding the software and engineering 
constraints under which the development occurred and 
helped provide focus in their feedback to the developers. 
Such a cooperative design approach was mutually bene- 
ficial to the controller and the engineer as both parties 
were able to satisfactorily understand each other’s needs 
throughout development. 

Early involvement of the air traffic controller was also 
extremely valuable for addressing human factors issues. 
The early and continuous involvement of controllers 
allowed interface and suitability issues to be tested and 
implemented throughout development. Issues such as 
information extraction, the assessment of workload, and, 
especially, the assessment of controller acceptance were 
able to be accommodated throughout development when 
changes could be made easily. Measurements taken by 
instruments such as the CARS have provided valuable 
insight into acceptance issues and also provided direction 
to the controllers on how to evaluate the system in the 
absence of outside guidelines. 

By implementing the process described in this paper, 
the controllers have become more than just end-users 
awaiting a product. They are actively a part of the devel- 
opment of FAST and they clearly have demonstrated their 
enthusiasm for its success. Their active participation has 
contributed to the FAA’s continued awareness and sup- 
port of CTAS development. In addition, the controllers 
have become strong advocates of FAST in their facility, 
working to promote FAST as a beneficial addition to the 
controller’s tools. Once the controllers themselves were 
satisfied that FAST was going to benefit the facility, and 
was not intended to replace the controller, they were able 
to provide reassurance to the facility that the concerns 
over job satisfaction and workload were being taken into 
consideration. Clearly, the input of controllers from the 
deployment site in early stages of development will be 
critical for upcoming operational testing. The controllers 
provided the developers with cues toward the overall 


facility culture, and have helped to prepare the facility for 
the introduction of new technology. 

Concluding Remarks 

The FAST development was a collaborative effort 
between air traffic controllers, developers, and human 
factors specialists. The development proceeded with 
increasing levels of expertise and complexity. The con- 
cept was first developed by engineers, then tested with 
retired controllers to gain early data on controller 
acceptability. Once this initial phase demonstrated 
benefits both to the controller and the air traffic control 
system, expert controllers from the initial deployment 
facility were introduced to gain specific expertise for the 
FAST system. These expert controllers drove the system 
development toward a usable system for their facility 
through real-time simulation and by shadowing live 
traffic operations with the FAST system. Finally, after a 
high degree of system fidelity was reached, a controller 
assessment team for the initial deployment site was 
brought into the final phases of development in order to 
prepare for an operational test. The assessment team 
controllers contributed to the final tuning of the system 
through real-time simulation and through the use of a new 
Controller Acceptance Rating Scale. 

The complexity in developing automation aids for ATC 
is due to more than the complexity of modeling ATC 
strategies. The successful introduction of automation aids 
into the air traffic control environment must consider the 
impact on the air traffic controller in the form of changes 
to controlling strategies, workload, and job satisfaction. 
Early controller involvement helps to identify such con- 
cerns at a stage when changes to the software and educa- 
tion for the controller and developer can easily take place. 
In addition, early controller involvement helps to increase 
acceptance, which is the key measure of the software’s 
success. 

If the controllers’ input is delayed until the end of the 
software development process, their sense of involvement 
is diminished and their operational concerns are likely 
to be ignored because of the inability to accommodate 
changes at later stages of development. In addition, pre- 
venting controllers from making inputs until the later 
stages of development assumes that the developers have 
anticipated all problems and that the facility doesn’t need 
any time to grow accustomed to potentially large changes 
to the nature of its work. 

By using a collaborative development approach, the 
FAST development team successfully integrated con- 
trollers into the development process. The controllers 
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became integral to the development as much for their 
ATC expertise as for their input into directing the intro- 
duction of automation into the field. The support of the 
controllers helped focus the development of FAST, and 
the controllers themselves have been instrumental in the 
push toward field testing. This motivation and the direc- 
tion provided by the controllers have helped the software 
become a tool that the controller will find useful, bene- 
ficial to the overall operations, and helpful for meeting 
the increasing demands of the ATC environment. The 
controllers’ cooperative participation with the engineers 
has become key to the successful development of auto- 
mation for the air traffic control system. 
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